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Abstract 

Background: This study examines research knowledge infrastructures (RKIs) found in health systems. An RKI is 
defined as any instrument (i.e., programs, interventions, tools) implemented in order to facilitate access, 
dissemination, exchange, and/or use of evidence in healthcare organisations. Based on an environmental scan (17 
key informant interviews) and scoping review (26 studies), we found support for a framework that we developed 
that outlines components that a health system can have in its RKI. The broad domains are climate for research use, 
research production, activities used to link research to action, and evaluation. 

The objective of the current study is to profile the RKI of three types of health system organisations-regional 
health authorities, primary care practices, and hospitals-in two Canadian provinces to determine the current mix of 
components these organisations have in their RKI, their experience with these components, and their views about 
future RKI initiatives. 

Methods: This study will include semistructured telephone interviews with a purposive sample region of a senior 
management team member, library/resource centre manager, and a 'knowledge broker' in three regional health 
authorities, five or six purposively sampled hospitals, and five or six primary care practices in Ontario and Quebec, 
for a maximum of 71 interviewees. The interviews will explore (a) which RKI components have proven helpful, (b) 
barriers and facilitators in implementing RKI components, and (c) views about next steps in further development of 
RKIs. 

Discussion: This is the first qualitative examination of potential RKI efforts that can increase the use of research 
evidence in health system decision making. We anticipate being able to identify broadly applicable insights about 
important next steps in building effective RKIs. Some of the identified RKI components may increase the use of 
research evidence by decision makers, which may then lead to more informed decisions. 



Background Multiple factors determine the nature and extent of 

Significant worldwide investments that are made in bio- engagement in KT by different stakeholder groups. A 

medical and health research are underutilised because of common challenge that all decision makers face relates 

challenges to knowledge translation (KT). KT is defined to the lack of knowledge management skills and infra- 

as 'the exchange, synthesis and ethically-sound applica- structure (i.e., the sheer volume of research evidence 

tion of researcher findings within a complex system of currently produced; access to research evidence; time to 

relationships among researchers and knowledge users to read; and skills to appraise, understand, and apply 

improve health' [1]. Evidence shows that health systems research evidence). Better knowledge management is 

frequently fail to optimally use research evidence, which necessary but probably insufficient to ensure effective 

leads to inefficiencies, reduced quantity and quality of KT given other challenges that may operate at different 

life for citizens, and lost productivity [2]. levels, including the healthcare system (e.g., financial dis- 
incentives), healthcare organisation (e.g., inappropriate 
skill mix, lack of facilities or equipment), healthcare 
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adherence to medical advice) [3]. Frequently, multiple 
challenges are present at different levels of the health- 
care system. A further challenge for KT relates to the 
dynamism and constant evolution of healthcare. As a 
result, KT approaches and activities need to recognise 
and keep pace with the changes in the healthcare sector. 

Physical and cognitive access to evidence are chal- 
lenges to KT for health system managers and policy 
makers. Three primary KT approaches target these 
groups: push, pull, and exchange efforts [4]. The push 
approach includes activities undertaken by researchers 
to package and disseminate research evidence outside 
the scholarly community. Pull activities focus on the 
efforts by health system managers and policy makers to 
access and use research evidence. Exchange activities 
focus on building and maintaining relationships between 
researchers and managers and policy makers. We pro- 
pose to study research knowledge infrastructures (RKIs) 
that build on push, pull, and/or exchange components. 

In this study, RKI is defined as any instrument {i.e., 
programs, interventions, tools, devices) implemented in 
key organisations and broader health systems of which 
they are a part (hereafter called health systems) in order 
to facilitate access, dissemination, exchange, and/or use 
of evidence. These instruments can be subdivided into 
two broad categories: (1) technological instruments {e.g., 
electronic databases, search engine) and (2) organisa- 
tional instruments {e.g., documentation specialists; data 
analysts, such as epidemiologists; knowledge brokers 
who, for example, manage the collaboration between the 
organisation and external information and knowledge 
producers; and training programs to help in, for exam- 
ple, searching for information, appraising information, 
adapting and using both statistical data and academic 
research) [5]. Multiple types of evidence can be used by 
health system managers and policy makers, including 
(among others) administrative information, legal infor- 
mation, academic research, financial data, and surveil- 
lance data. In this project, we will be focusing on access 
to and utilisation of the following types of evidence: aca- 
demic research outputs {i.e., articles, research reports, 
and books) and population and health system data {i.e., 
surveillance data, service utilisation data, and other non- 
financial performance data). 

Findings from largely observational studies suggest 
that a number of factors are associated with the use of 
research evidence by policy makers, including personal 
contact with researchers, timeliness and relevance of 
research, and inclusion of summaries with recommenda- 
tions [6,7]. These and other findings have led to the 
development of a number of KT approaches targeting 
policy makers and senior health system managers [8,9]. 
Most of these approaches have a strong theoretical basis 
and face validity. We developed an RKI framework that 



identified potential RKI components that a healthcare 
system or a hospital could have in its RKI. The frame- 
work includes the broad domains of Climate for 
Research Use, Research Production, Activities Used to 
Link Research to Action, and Evaluation. Table 1 
includes the detailed elements within each RKI domain. 
We conducted an environmental scan and scoping 
review to conduct a preliminary assessment of possible 
RKI components that exist in various countries and in 
the applicable literature. 

We conducted an environmental scan of current RKIs 
in two Canadian provinces {i.e., extraction of key infor- 
mation from organisations' websites and informal inter- 
views with key informants). Our informal interviews 
with 17 key informants were aimed at generating exam- 
ples of tools, interventions, and components of RKIs. 
With respect to the RKI domain of Climate for Research 
Use, many of the informants referred to their organisa- 
tion's strategic plan, which explicitly supported the use 
of research evidence in decision making. Most of the 
informants reported having existing relationships with 
researchers, and formal relationships were often facili- 
tated by the structure of the organisation. In the RKI 
domain of Research Production, some informants sug- 
gested the existence of priority-setting processes, and 
although research production appears limited within 
most organisations, some suggested they have the capa- 
city to commission research. Efforts to disseminate 
research findings both within and across organisations 
and the usage of knowledge brokers were mentioned 
when discussing push activities. 

Many of the key informants mentioned that the cate- 
gories within the Facilitating Pull domain enabled them 
to access research evidence to be used in decision mak- 
ing {i.e., organisations benefited from full access to bib- 
liographic databases through partnerships, some 
organisations have comprehensive library services, and 
some have information tools to assist management). 
Within the Pull Activities domain, two key informants 
identified the establishment of rapid response units that 
produced information syntheses within approximately 
two weeks for managers, decision makers, and policy 
makers. Furthermore, various forms of training pro- 
grams were discussed that educated participants in the 
use of research evidence in decision making. No exam- 
ples of the Evaluation of Efforts to Link Research to 
Action were provided. The environmental scan demon- 
strates that health system managers and policy makers 
in Ontario and Quebec have instituted in their organisa- 
tions and in the health system as a whole varying com- 
ponents of an RKI; yet, the breadth, depth, usage, and 
effectiveness of these efforts have yet to be determined. 

We also conducted a scoping review as a preliminary 
assessment of the size and scope of the available 
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Table 1 Possible organisational level components of a research knowledge infrastructure (RKI) 

Domain Elements 

I. Climate for Research Use • Accreditation acknowledges and rates the use of research evidence in decision making 

• Mission, vision, values, and strategic plan reflect the value placed on the use of research evidence in 
decision making 

• Structures or positions exist within the organisations to ensure accountability for using research evidence 
in decision making 

• Clear points of contacts within organisations regarding where to turn to obtain research evidence 

• Formal and informal relationships with people outside the organisation who can assist in obtaining the 
appropriate research evidence 

• Recruitment and retention strategies that reflect the value of the use of research evidence in decision 
making 

• Recognition of employees who use research evidence within the organisation 

II. Research Production • Participating in regular priority setting processes for the research evidence needed to meet managerial 

and policy-making needs 

• Ensuring that the appropriate research commissioning capacity is in place to commission or execute the 
research if it is deemed high priority yet no applicable research is available 



II. Activities Used to Link Research to 
Action (4 parts) 



Ilia. Push Efforts 


• Knowledge intelligence service that scans the literature and distributes relevant research evidence 
throughout the organisation 

• Individual(s) responsible for identifying teaching moments to profile research evidence 


lib. Facilitating Pull Efforts 


• Enabling 'easy access' to research evidence through physical tools (e.g., information technology systems) 

• Enabling 'easy access' to research evidence through appropriate resources (e.g., websites with optimally 
packaged reviews) 


lllc. Pull Efforts 


• Summarising or conducting primary research via a rapid response unit that supports the use of research 
in decision making 

• Decision-making processes that promote the use of research in decision making 

• Self-assessment tools that focus on capacity to find and use research evidence in decision making 

• Training and continuing education that focus on finding and using research evidence in decision making 

• Use of dedicated staff to 'pull' research into decision making 


lid. Exchange Efforts 


• Regular meetings that highlight relevant research 




• Interactive workshops that focus on the use of research in decision making 


IV. Evaluation of Efforts 


• Monitoring and evaluation efforts on the use of research in decision making 



literature on the effectiveness of particular models, or 
specific components, of RKI, as well as regarding the 
problems they are addressing, other features of RKIs 
that may influence their design, and related implementa- 
tion considerations. We manually reviewed (a) http:// 
healthsystemsevidence.org, a continuously updated repo- 
sitory of syntheses of research evidence about govern- 
ance, financial, and delivery arrangements within health 
systems and about implementation strategies that can 
support change in health systems; (b) a database of stu- 
dies used in a systematic review of the factors that influ- 
ence the use of research evidence in public policy 
making; and (c) other studies or reports that we were 
aware of that were applicable to our scoping review. In 
reviewing these databases, we were trying to identify all 
primary studies that address RKI either in whole or in 
part. 

We were unable to identify any studies evaluating the 
effects of a full RKI on the use of evidence by health 
system managers and policy makers; however, the 



scoping review did uncover 25 qualitative studies and 
one randomised control trial that addressed different 
components of the RKI framework (for a list of included 
studies, please contact MEE). The majority of the stu- 
dies (25 studies) addressed categories within the Climate 
for Research Use domain, while the next most addressed 
domains were Activities Used to Link Research to 
Action, Facilitating Pull, and Pull Activities (16 studies). 
Fourteen studies evaluated the Research Production 
domain, and 10 studies addressed exchange activities. A 
minority of the studies (seven studies) reported on Push 
Activities, and even fewer (two studies) reported on the 
Evaluation of Efforts to Link Research to Action. The 
one subdomain that was reported most often, by 20 stu- 
dies, as influencing the use of research in decision mak- 
ing was the formal and informal relationships to people 
outside the organisation who can assist in obtaining the 
appropriate research evidence (see Additional File 1 for 
illustrative examples associated with each category). 
This scoping review illustrates that while numerous 
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independent initiatives are in place in different health- 
care systems, no one RKI framework has been developed 
or studied to determine overall effectiveness. 

Building on the environmental scan and scoping 
review, we propose to conduct interviews with health 
system managers and policy makers in Ontario and 
Quebec to determine which RKI components have actu- 
ally been implemented and are perceived to have added 
value in their organisations, the barriers and facilitators 
to implementing these RKI components, and views 
about how to further support RKI development. The 
RKI project will be a springboard to cross-organisation 
and cross-system research to better understand how to 
match particular RKI elements to different contexts. 

Objectives 

The objectives of our study are as follows: 

♦ To profile the RKI of three key types of healthcare 
organisations in two Canadian provinces (Ontario and 
Quebec): regional health authorities, primary care prac- 
tices, and hospitals 

♦ To identify barriers and facilitators to implementing 
various RKI components 

♦ To determine what these organisations view as 
important next steps in building effective RKIs 

Methods 

We will conduct in-depth, semistructured telephone 
interviews with a senior management team member (who 
will be focused more on organisational infrastructure), 
library/resource centre manager (who will be focused 
more on technical infrastructure), and (for some types of 
large organisations) a knowledge broker. These inter- 
views will be conducted in three regional health authori- 
ties (three participants interviewed in each, for a total of 
18 interviews), five or six purposively sampled hospitals 
(depending on the hospital's size, we will interview two 
or three participants in each hospital, for a total of 29 
interviews), and five or six primary care practices (with 
two individuals interviewed in each organisation, for a 
total of 10 to 12 interviews) in Ontario and Quebec. 

We are interested in examining organisations that 
have already participated in strategic behaviour with 
respect to RKI (or that are linked to such organisations). 
We identified a sample of these organisations by exam- 
ining the publicly available list of participants that have 
been a part of the Canadian Health Services Research 
Foundation's Executive Training for Research Applica- 
tion program. This program is geared towards health 
system managers and policy makers in Canadian health 
system organisations and aims to develop capacity and 
leadership to optimise the use of research evidence. The 
organisations from which these participants are drawn 
committed to having one or more of their senior 



executives enroll in the program and participate in four 
1- to 2-week training sessions over a two-year period, 
supporting their executive(s) in designing and executing 
an evidence-based 'intervention project', and sending 
their Chief Executive Officer to a final meeting where 
the intervention project would be discussed. We aug- 
mented the sample based on our knowledge of other 
organisations that have demonstrated strategic beha- 
viour with respect to RKI. 

The focus of the interviews will be to explore which 
RKI components have been implemented and are per- 
ceived to have added value in the participant's organisa- 
tion (and which not), the barriers and facilitators 
encountered in implementing these RKI components, 
and views about next steps in supporting the further 
development of RKIs. Semistructured interviews were 
utilised because they allow the interview participants to 
respond freely and focus on the area of the framework 
that the respondents think most apply to their organisa- 
tion [10]. Furthermore, using semistructured interviews 
permits the interviewer to probe issues that may be of 
interest to the current research but that are not specifi- 
cally addressed by the interview guide [11]. 

We will tape and transcribe all interviews, and they 
will be verified by the interviewer before analysis begins. 
NVivo, Version 9, (QSR International, Cambridge, MA) 
will be used for data management. A constant compara- 
tive method for the thematic analysis of the interview 
data will be used (with one very experienced qualitative 
researcher working with one doctoral student to identify 
themes). Interviews will be analysed in clusters based on 
their organisations, meaning all interviews from one 
organisation will be analysed in tandem, followed by all 
other institutions within that organisational context (i.e., 
regional health authorities, hospitals, and primary care 
practices). First we will read the entire interview to get a 
sense of the whole interview and initial impressions. 
Then we will read the text a second time, code units of 
text, and compare initial codes. Coded segments will 
then be reanalysed, coded into subcategories, and com- 
pared again. Two researchers will constantly compare 
the coding during each stage. 

Discussion 

Upon completion of the qualitative study, we will pro- 
duce a report that describes the RKI status in regional 
health authorities, hospitals, and primary care practices; 
identifies barriers and facilitators to implementing var- 
ious RKI components; and provides an overview of what 
these health system managers and policy makers view as 
important next steps in building effective RKIs. Based 
on these qualitative findings, we will develop a protocol 
for a cross-sectional survey that will generate more gen- 
eralisable information. 
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The environmental scan and scoping review have 
identified key elements of RKI, and this qualitative work 
will explore the extent of the use of these elements in 
different settings. We anticipate being able to identify 
broadly applicable insights about important next steps 
in building effective RKIs. This is the first qualitative 
examination of potential RKI efforts that can increase 
the use of research evidence in health system decision 
making. Some of the identified RKI components may 
increase the use of research evidence by decision 
makers, which may then lead to more informed deci- 
sions, and hopefully to a strengthened health system 
and improved health. 

Additional material 
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Additional file 1: Illustrative examples of quotes from the scoping 
review associated with the research knowledge infrastructure 
categories. A scoping review was conducted looking at all relevant 
literature to determine if the research knowledge infrastructure (RKI) 
framework that we developed was supported by the empirical literature. 
This additional material contains excerpts from the scoping review that 
demonstrate quotes associated with the appropriate RKI categories [12- 
25], 



